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Abstract 

We consider online competitive algorithms for the problem of collecting weighted items from 
a dynamic set S, when items are added to or deleted from S over time. The objective is 
to maximize the total weight of collected items. We study the general version, as well as 
variants with various restrictions, including the following: the uniform case, when all items 
have the same weight, the decremental sets, when all items are present at the beginning and 
only deletion operations are allowed, and dynamic queues, where the dynamic set is ordered and 
only its prefixes can be deleted (with no restriction on insertions). The dynamic queue case is a 
generalization of bounded-delay packet scheduling (also referred to as buffer management). We 
present several upper and lower bounds on the competitive ratio for these variants. 

1 Introduction 

Whac-a-mole is an old arcade game, where plastic "moles" pop out of holes in the machine for 
short periods of time, in some unpredictable way, and the player uses a mallet to "whack" as many 
moles as possible0 In the generalized version that we consider, multiple moles may be present at 
the same time, and different moles may have different values. 

In a more formal setting, we think of it as a dynamic set S of weighted items (moles). Before 
each step, some items can be deleted from S and other items can be added to S. We are allowed 
to collect one item (whack one mole) from S per step. Each item can be collected only once (the 
mallet does its job). The objective is to maximize the total weight of the collected items. 

To our knowledge, this simple and fundamental problem has not been explicitly addressed in 
the literature. By placing appropriate assumptions on the structure of S or on the type of allowed 
operations, one can obtain a number of natural special cases, some of which are related to known 
online problems. 

We study the general case of dynamic sets, as well as some restricted cases, among which we 
focus on the following versions: 

Dynamic Queue: In this case, S represents a list, i.e. the items in S are ordered. Items can be 
added to S at any place, but only a prefix of S can be deleted. A queue is called a FIFO queue if 
insertions are allowed only at the end. 
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Decremental Sets: Here, all items are added at the beginning, and only deletions are allowed 
afterwards. In particular, one can consider the case of decremental queues, where only prefix- 
deletion operations are allowed. 

The case of dynamic queues generalizes the well-studied problem of bounded-delay packet 
scheduling (a.k.a. buffer management), or, equivalently, the problem of scheduling unit jobs with 
deadlines for maximum weighted throughput. In this problem, packets with values and deadlines 
arrive in a buffer of a network link. At each step, we can send one packet along the link. The 
objective is to maximize the total value of packets sent before their deadlines. This is a special case 
of our dynamic queue problem, where packets are represented by items ordered according to dead- 
lines. The difference is crucial though: in packet scheduling, packet arrival times are unknown but 
their departure times are known, while in dynamic queues both the arrival and departure times are 
not known. The FIFO case generalizes the version of packet scheduling with agreeable deadlines. 

Competitive algorithms for various versions of bounded-delay packet scheduling problem have 
been extensively studied [H [H [8j [101 [12] . In particular, it is known that no deterministic online 
algorithm can have competitive ratio better than <f> ~ 1.618 [HIS], and algorithms with competitive 
ratio ~ 1.83 have been recently developed |12} [6], These are the best lower and upper bounds for 
this problem and closing this gap remains a tantalizing open problem. For agreeable deadlines, an 
upper bound of 4> has been established in [TT] . 

The dynamic set problem has some indirect connections to other packet scheduling problems, 
where the objective is to maximize the value of packets reaching their destination under various 
scenarios EJ El HD] . 

A different, metric version of whac-a-mole was considered in [7]. In this approach moving the 
mallet to a mole takes time and the duration of moles' exposure is known. However, their results 
are inapplicable in our model. 

Our results. We first consider the general case of dynamic sets. For deterministic algorithms, 
it is quite easy to show a lower bound of 2 (even for two items in the decremental case), and this 
ratio can be achieved by a simple greedy algorithm that always collects the heaviest item. 

For randomized algorithms, we focus on the uniform case, when all items have weight 1. We 
show that no memoryless randomized algorithm can achieve competitive ratio better than 2. We 
then study the uniform decremental case, for which we give an online randomized algorithm with 
competitive ratio e/(e — 1) (against an oblivious adversary) and prove a matching lower bound. 

Most of our results concern dynamic queues. In the deterministic case, even for decremental 
queues, it is quite easy to show a lower bound of <\>. We improve this bound, by proving a lower bound 
of ~ 1.63, that applies even to the decremental case. We also show that no memoryless algorithm 
can have a ratio smaller than 2. This contrasts with the result of Englert and Westerman [6] 
who gave a memoryless algorithm for packet scheduling with ratio ~ 1.893. Thus, at least for 
memoryless algorithms, knowing the exact deadlines helps. 

As for upper bounds, we give the following deterministic algorithms: (i) A 1.737-competitive 
algorithm for decremental queues (beating the ratio 2 of the naive greedy algorithm), (ii) A 
1.8-competitive algorithm for FIFO queues, (iii) A (^-competitive algorithm for dynamic queues 
when the item weights are non-decreasing (w.r.t. their position in the queue). This last result has 
implications for packet scheduling, as all lower bound proofs for that problem use instances where 
packets' weights increase with deadlines. Thus, either the ratio eft for the packet scheduling problem 
can be achieved, or, to improve the lower bound, one would have to use non-monotone instances in 
the proof. Our competitive analysis uses a novel invariant technique involving dominance relations 
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between sets of numbers, and is likely to find applications in the analysis of packet scheduling. 

Finally, we address the case of dynamic queues and randomized algorithms. The algorithm RMix 
[I] can be easily adapted to dynamic queues, achieving competitive ratio of e/(e — 1) against 
an adaptive adversary. (In spite of the same value, this result is not related to our analysis of 
the uniform dynamic sets.) We prove a matching lower bound, which says that no memory less 
randomized algorithm for dynamic queues can have competitive ratio smaller than e/(e — 1). 

Due to space limitations some proofs are presented in the appendix. 

2 Preliminaries 

We refer to the items currently in S as active. In other words, those are the items that have been 
already inserted but not yet deleted. An item is called pending for an algorithm A if it is active 
but not yet collected by A. We denote the weight of item x as w x and the total weight of a set of 
items X as w(X). 

An instance of the problem is defined by a sequence / of item insertions or deletions. A solution 
consists of a selection sequence that specifies items selected at each step. An optimal solution is 
computable in polynomial-time: represent the instance as a bipartite graph G whose partitions are 
items and time steps. Item a is connected to the time steps when a is active with edges of weight 
w a . The maximum- weight matching in G represents an optimal collection sequence. 

When S is a dynamic queue, S is represented by a list. We use symbol "<" to represent the list 
ordering, i.e. a < b means that a is before b in the list. In case of dynamic queues, optimal solutions 
have special structure that we explore in our competitiveness proofs, namely they satisfy (w.l.o.g.) 
the following Earliest- Expiration- First (EEF) Property: If a, b are active at the same time, a < b, 
and both a, b are collected, then a is collected before b. 

An online algorithm A is called 1Z- competitive, if for any instance /, the gain of A on / is at 
least the optimum gain on / divided by 1Z. (An additive constant is sometimes also allowed in this 
bound; in all our upper bounds this constant is 0, as the weights can be scaled up.) The competitive 
ratio of A is the smallest 1Z for which A is 7£-competitive. 

3 Dynamic Sets 

For general dynamic sets, it is easy to show the lower and upper bounds of 2 on the competitive 
ratio of deterministic online algorithms. To prove the lower bound, the adversary can start with 
two items, say a and b with w a = ioj, = 1. Assume, by symmetry, that the algorithm first collects 
a. Then the adversary can collect b in the first step, delete it, and collect a in the next step, while 
the algorithm has no items to collect. 

This bound can be achieved by a simple Greedy algorithm: at each step, if there is at least one 
pending item, collect the maximum-value pending item. 

Fact 1 Greedy is 2-competitive for dynamic sets. 

Proof: The proof is a straightforward adaptation of the proof for packet scheduling [HI [9], we 
include it for completeness sake. The items collected by the adversary are charged to the items 
collected by Greedy. Suppose the adversary collects an item b at time t. If this item was collected 
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at time t or earlier by Greedy, we charge it to that item in G reedy 's sequence. Otherwise, we charge 
it to the item collected by Greedy at time t. 

Let a be an item collected by Greedy at some time u. It receives at most two charges: one from 
itself, if it was collected by the adversary at time u or later, and the other one from the item b 
collected by the adversary at time u. If a receives a charge from b, then b is pending for Greedy at 
time t, and therefore Wb < w a . Therefore the charge to a is at most 2w a . 

Summarizing, all adversary items are charged to our items, and each our item receives a charge 
of at most twice its weight. Thus Greedy is 2-competitive. □ 

Now we turn our attention to randomized algorithms. For the adaptive adversary it is not hard 
to show a lower bound of 2. The adversary strategy is this: issue n items of weight 1. Collect any 
item a that is collected by the algorithm with probability at most 1/n. Let b be the item collected 
by the algorithm. If b = a, remove all items. If b 7^ a, remove all items except b and collect b. With 
probability at most 1/n, we have b = a and both the algorithm and the adversary collect b. With 
probability at least 1 — 1/n, the algorithm gets one item and the adversary gets two items. So the 
ratio is arbitrarily close to 2. 

For oblivious adversaries we concentrate on the uniform decremental case. We show that for 
any randomized algorithm the optimal strategy for the adversary is to repeatedly take and remove 
an active item. We consider UniRand algorithm, which at each step collects one of the pending 
items with equal probability. 

For the lower bound we use the Yao min-max principle and consider inputs which are constructed 
by the following random process: at each step choose an active item uniformly at random, collect 
it, and delete it after the step. 

Let E a>p be the expected number of items collected by an algorithm if a items are active and, 
among them, p items are pending. It can be shown that both in UniRand analysis and in the lower 
bound construction, E a>p satisfies the following recursive formula: 

E a ,o = 0, E a \ = 1, E ap = • E a -i lP -i + • + 1 • 

a a 

The unique solution of this formula can be estimated by E a ,p ~ a(l — (1 — l/a) p ). Using exact 
approximations of E a>p , we may prove the following theorems. 

Theorem 2 The competitive ratio of any randomized algorithm for the uniform case of decremental 
sets is at least e/(e — 1). 

Theorem 3 Algorithm UniRand is e/(e — 1)- competitive for the uniform case of decremental sets. 

4 Dynamic Queues — Deterministic Algorithms 

In this section we consider deterministic online algorithms for the case when S is a dynamic queue, 
that is, S is an ordered list and only a prefix of S can be deleted. In the fully dynamic case, items 
can be inserted anywhere, while in the decremental case, all items are inserted at the beginning. 

Fact 4 Every deterministic algorithm for queues has competitive ratio at least <fi ~ 1.618, even for 
decremental queues and with items of "life span" at most 2. 
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Proof: We start with two items in the list, a < b, with values w a = 1 and u>& = 0. If the algorithm 
chooses b, the adversary chooses a and deletes it, and in the next step he chooses b, so the ratio is 
(w a + Wb)/wb = (1 + 4>)/4> = (/>• If the algorithm chooses a in the first step, the adversary chooses 
b and deletes both items, so the ratio is Wb/w a = 4> again. □ 

This section includes some lower and upper bounds for the competitive ratio of deterministic 
online algorithms. Starting with lower bounds, we first prove that no deterministic algorithm for 
the dynamic queue — in fact, even for the decremental case — can achieve a competitive ratio better 
than 1.63. For memoryless algorithms we give a lower bound of 2. 

Our first upper bound concerns decremental queues, for which we present a deterministic online 
algorithm DecQueEFH with competitive ratio ~ 1.737. Next, we present a deterministic online 
algorithm FIFOQueEH that achieves competitive ratio 1.8 for FIFO Queues. We conclude this 
section with the algorithm MarkAndPick that achieves competitive ratio </> for dynamic queues in 
which the item weights are non-decreasing (this also gives a ^-competitive algorithm for scheduling 
packets with non-decreasing weights). 

4.1 Lower Bounds for Decremental Queues 

We prove that no online deterministic algorithm can have a competitive ratio smaller than 1.63 for 
decremental queues. The proof is by presenting an adversary's strategy that forces any deterministic 
online algorithm A to gain less than 1/1.63 times the adversary's gain. 

Adversary's strategy. To get a cleaner analysis, we first present the argument for the dynamic 
case (with insertions allowed), and explain later how it extends to the decremental case. We assume 
that the items appear gradually, so that at each step the algorithm has at most three items to choose 
from. 

Fix some n > 2. To simplify notation, in this section we refer to items simply by their weight, 
thus below "V denotes both an item and its weight. The instance consists of a sequence of 2n 
items 1, z\, z 2 , z 2n - 2 , z 2n (note that there is no item indexed 2n — 1) such that 

z 2 < Z4 < ••• < z 2n -2 < z 2n < z 2n -3 < ••• < ^3 < Z\ < 1, and 

l> Zi> Z 2 > ... > Z 2n -3 > Z 2n -2 > Z 2n > 0. 

The even- and odd- numbered items in this sequence form two roughly geometric sequences. In fact, 
z 2 i is only slightly smaller than z 2 i-i, for all i = 1, ...,n — 1. 

Initially, items Z2 < z\ < 1 are present. In step i = 1,2, ...,n — 1, the adversary maintains the 
invariant that the active items are z 2 i < z 2 i-i < — < 1, of which only three items z 2 i, z 2 i-i and 1 are 
pending for A (i.e. A already collected z 2 is, ...,z\). The adversary's move depends now on what 
A collects in this step: 

(i) A collects z 2 {. Then the adversary ends the game by deleting all active items. In this case the 

adversary collects i heaviest items: 1, z\, z 2 , £3, . . . , 

(ii) A collects 1. The adversary ends the game by deleting x 2 i and x 2 i-\. This leaves A with no 

pending items, and the adversary can now collect A's items one by one. Overall, in this case 
the adversary collects 2i heaviest items: 1, z±,z 2 , . . . , z 2 i_ 2 , z^i-i- 
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(iii) A collects Zi%-\- In this case the game continues. If i < n — 1, the adversary deletes z^i, 
inserts Z21+2 and Z21+1 into the current list (according to the order defined earlier), and we 
go to step i + 1. The case i = n — 1 is slightly different: here the adversary only inserts the 
last item Z2 n before proceeding to step n (described below). 

If the game reaches step n, A has two pending items, Z2 n and 1. In this step, the adversary 
behavior is similar to previous steps: if A collects Z2 n , then the adversary deletes the whole sequence 
and collects n heaviest items: 1, z±, Z2, 23, ... , z n -\. \iA collects 1, the adversary deletes z n , leaving 
A without pending items, and allowing the adversary to collect the whole sequence. 

Our goal is to find a sequence {zi}, as above, and a constant 1Z such that 



Lemma 5 Suppose that there is a sequence 1, z±, £2n— 2> Z2 n , and a constant 1Z that satisfy in- 
equalities (Q]) and (0). Then there is no 1Z- competitive deterministic online algorithm for dynamic 
queues, even in the decremental case. 

The lemma should be clear from the description of the strategy given earlier, since the sums in 
inequalities ([T]) and ([2]) represent the gains of the adversary and the algorithm in various steps. The 
only part that needs justification is that the lemma holds in the decremental case. To see this, we 
slightly modify the adversary strategy: The sequence {z{\ is created all at once in the beginning, 
and whenever A deviates from the choices (i), (ii), (iii), it must be collecting an item lighter than 
Z2i, and thus the adversary can finish the game as in Case (i). 

Lemma [5] and straightforward calculations for n = 3 (6 items) yield the following. 

Theorem 6 There is no deterministic online algorithm for dynamic queues (even in the decre- 
mental case) with competitive ratio smaller than 1.6329. 

A natural question arises how much this bound can be improved with sequences {z{\ of arbitrary 
length. For n = 5 (10 items) one can obtain 1Z = 1.6367... and our experiments indicate that the 
corresponding ratios tend to ~ 1.6378458, so the improvement is minor. However, it is easy to 
prove a lower bound of 2 for memoryless algorithms, i.e. algorithms that decide which item to 
collect based only on the weights of their pending items. 

Theorem 7 For dynamic queues, no memoryless algorithm has competitive ratio smaller than 2. 
4.2 Upper Bound of 1.737 for Decremental Queues 

Algorithm DecQueEFH: The computation is divided into stages, where each stage is a single 
step, a pair of consecutive steps, or a triple of consecutive steps. By h we denote the maximum- 
weight pending item from the first step of the stage. We use two parameters, (5 = (\/l3 + l)/8 
and £ = (\/T3 + l)/6. Note that j3 < £. Without loss of generality, we assume that there are 
always pending items, for we can always insert any number of 0-weight items into the instance, 
without changing the competitive ratio. In the pseudo-code below, we assume that after each item 
collection the algorithm proceeds to the next step of the process. 



ft- (^2j+2 + Ei=l 2; 2^l) < 1 + Ei=l^ 



for all < j < n 
for all < j < n 



(1) 
(2) 
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(E) let h be the heaviest pending item 

collect the earliest pending item e with w e > f3wh 

(F) if h is not pending then end stage and goto (E) 
collect the earliest item / with Wf > £,Wh 

(H) if h is not pending then end stage and goto (E) 
collect h 

end stage and goto (E) 

Theorem 8 For decremental queues, the competitive ratio o/DecQueEFH is at most 1Z = 2(\/l3 — 
l)/3 « 1.737. 

Proof: We fix an instance and we compare DecQueEFH's gain on this instance to the adversary's 
gain. We assume (w.l.o.g.) that the adversary has the EEF property. 

The proof is by amortized analysis. We preserve the invariant that, after each stage, each 
item i that is pending for the adversary but has already been collected by DecQueEFH has credit 
associated with it of value equal uii . The adversary's amortized gain is equal to his actual gain plus 
the total credit change. To prove the theorem, it is then sufficient to prove the following claim: 

(*) In each stage the adversary's amortized gain is at most 1Z times DecQueEFH's gain. 

To prove (*), we consider several cases depending on the number of steps in a stage and on 
relative location of items collected by the adversary and by DecQueEFH. We assume that at each 
step the adversary collects items that were not collected by the algorithm before or during this 
stage. Otherwise, either the adversary collects an item that has been collected by DecQueEFH 
earlier and has credit on it, or DecQueEFH collected this item in this stage, in which case we can 
think of the algorithm giving the adversary credit for this item anyway (even though it is not 
needed, for this item is not pending for the adversary anymore). 

We first observe that e <! / <h (if / is defined for this stage). This follows immediately from 
/3 < £ < 1. An important consequence of this, that plays a major role in the argument below, is 
that when h is deleted, then e and / are deleted as well. 

Case 1 : h is not pending in (F). The stage has only one step, and the algorithm collects an item e 
with w e > (3wh- Let a be the item collected by the adversary. 

If h was deleted after (E) then e was deleted as well. Thus we do not give the adversary credit 
for e, and his amortized gain is w a < w^. The ratio is 



Suppose that h was collected (i.e. e = h). If a < h then w a < (3wh, so, together with the credit for 
h, the amortized adversary's gain is w a + Wh, and the ratio is 

Wa + Wh < l + P < K. 

w h 

If h < a, we need not give the adversary any credit, so the ratio is at most 1. 

Case 2 : h is pending in (F) but is not pending in (H). The stage has two steps, and we collect e 
and /, gaining w e + Wf > ((3 + £,) w h- The adversary collects two items, say a and b with a < b. 
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If h was deleted after (F), then both e and / are deleted as well, so we do not give the adversary 
any credits. Thus the adversary amortized gain is w a + Wb < 2wh- hi this case the ratio is 

< A = 4(7l3-l)/7 < U. 
w e + w f ~ f3 + i " 

Suppose now that the algorithm collected h in (F) (i.e. / = h), and thus our gain is actually 
w e +Wh > (l+j3)wh- If b<e, then the adversary amortized gain is w a +Wb+w e +Wh < (2(3+l)wh+w e . 
Thus the ratio is 

w a + w b + w e + w h < 3|+1 = 4(V j3 + 10)/17 < ^ 

If e < b < /i, the adversary's amortized gain is u; + w b + < (2£ + l)io/i, so the ratio is 

+ + < f + 1 = 2(5^13 + 23)/51 < ft. 

If /t < b, we need not give the adversary any credit, gaining w a + Wb < 2wh- So the ratio is less than 
the one above because 2£ + 1 > 2. 

Case 3 : h is still pending in (H). In this case the stage has three steps and we collect e, /, and h, 
for the total gain of w e + Wf + Wh > ((3 + £ + l)wh- The adversary collects three items a <\ b <] c 
and may get credit for some items. 

If c <] e, then the adversary gets credit for e, / and /i, and his amortized gain is w a + Wb + w c + 
w e + Wf + Wh < 3(3wh + w e + w f + w^. So the ratio is 

W a + Wb+Wc + We+Wf + Wh . 4/? + £ + l 

< 7 = /V-. 

I0 e + Wf + 10ft /3 + £ + l 

If e<c<f, then the adversary gets credit for /, /i, and his amortized gain is w a +Wb+w c +w f+Wh < 
3£,Wh + w f + Wh. So the ratio is 

W a + W b + W c + Wf + W h < 4^ + 1 _ 

If / < c < then he only gets credit for h, so his amortized gain is w a + Wb + w c + Wh < 4^. The 
ratio is 



w a + w b + w c + w h 



< -5 — - — t = 8(31 - 7>/l3)/27 < ft. 



io e + Wf + u)/j /3 + £ + l 

Finally, when h < c, the adversary amortized gain is u> a + + w c < 3w^, which is less than in the 
previous case. □ 

We attempted to extend the idea of Algorithm DecQueEFH to stages with more steps, but 
according to numerical experiments we conducted, this does not improve the competitive ratio. 
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4.3 Upper Bound of 1.8 for FIFO Queues 

We now extend the idea of the previous algorithm to FIFO queues. The algorithm uses two 
parameters, a and (3. The main idea is this: If the heaviest item h! from the previous step is no 
longer pending, this is fine and it simply begins another step. If h! is still pending and new items 
have been inserted to the queue, the algorithm inspects them. If the heaviest new item h is not 
too heavy (i.e., if awh < Wh 1 ), the algorithm ignores new items and collects h! . If h is very heavy 
(awh > uih'), the algorithm forgets about h! , resets heavy to h and collects the earliest pending 
item e, such that w e > /3wh- 

Algorithm FIFOQueEH: By h we denote the maximum- weight pending item and by h' the pre- 
vious maximum- weight pending item (initially h' is an imaginary item of weight 0). We use two 
parameters, < a,/3 < 1. Without loss of generality, we assume that there are always pending 
items, for we can always insert any number of 0-weight items into the instance, without changing 
the competitive ratio. In the pseudo-code below, we assume that after each item collection the 
algorithm proceeds to the next step of the process. 

let h be the heaviest pending item and h! the previous heaviest item 
(E) if (h! is not pending) or {h! is pending and aw^ > wy) then 

collect the earliest item e with w e > (3wh 
(H) else collect h! 

Theorem 9 The competitive ratio of FIFOQueEH with a = | and (3 = | is at most 1.8. 

By inspecting the decremental queue instance a <\ b <a c < d, with weights w a = Wb = (3 — e, 
w c = (3, Wd = 1, and weights w a [3, Wf, = w c = 1 — e, W4 = 1, one can observe the following. 

Theorem 10 The competitive ratio of Algorithm FIFOQueEH is at least 1.8, regardless of the choice 
of a and (3. 

4.4 Upper Bound for Non-Decreasing Weights 

In this section we give an online algorithm MarkAndPick that is (^-competitive for dynamic queues 
when item weights are increasing. More precisely, if < denotes the ordering of the items in the 
queue, then we assume that, at any time, for any two active items a, b G S, if a <l b then w a < Wb- 

Algorithm MarkAndPick: 

At each step £ = 1,2, ... 

if there is no pending item then wait, else 

let h be the heaviest unmarked item (not necessarily active) 

mark h, collect the earliest pending item i with Wi > Wh/cf) 

Basic idea: The proof is based on a charging scheme, where the items collected by the adversary 
are charged to our items in such a way that each our item is charged at most </> times its weight. In 
other words, each our item i has a budget <j)Wi and it uses its budget to pay for some items in the 
adversary's set. In fact, each such i pays for either one or two adversary's items. The charging is 
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done in two steps: (1) we charge the adversary items to the marked items, and (2) we charge the 
marked items to the algorithm's items. 

When we mark an item h, we collect an item i with Wi > Wh/4>, so its budget cftuii is sufficient 
to pay for h. In a simple scenario, if we collect items in all steps, we can afford to pay for all items 
marked in step (2). In step (1), we also show that the weight of the marked items exceeds that of 
the adversary, and these two facts easily imply (^-competitiveness. 

In reality, the situation is more complicated — in some moves MarkAndPick does not have any 
items to collect. For instance, suppose that the adversary collects an item f in such a step. This 
item is already collected by the algorithm and is now marked. Its budget is <jyWj>, and it pays for 
its mark, as well as for another item collected by the adversary in the past. Roughly, this is the 
item collected by the adversary when the algorithm was marking j'. 

The principle is, that any idle step is a consequence of a step in which the algorithm marked 
h = j' , collected an item i, and the adversary collected an item j smaller than i and still pending 
for the algorithm. (This is not exactly correct, but it reflects the main principle. It may happen 
that the marked item h "responsible" for the idle step in which the adversary collects j' is actually 
different from f, but later in the process h "transfers" this responsibility to j'.) In that case, 
w j < Wj'/(j). We refer to such j's as "extra" items (even though those are not exactly the extra 
moves, but they cause extra moves later). This idea can be formalised and proved rigorously. 

Theorem 11 Algorithm MarkAndPick is (/)- competitive for dynamic queues if item weights are 
non- decreasing. 

5 Dynamic Queues — Randomized Algorithms 

In this section, we consider randomized algorithms for dynamic queues. Chin et al. [4] designed 
a randomized algorithm RMix for bounded-delay packet scheduling. This algorithm is memoryless 
and achieves competitive ratio e/(e — 1) against an adaptive adversary. The competitiveness proof 
for RMix applies, with virtually no changes, to dynamic queues. In this section we show that this 
bound is tight. 

Theorem 12 For dynamic queues, any memoryless randomized algorithm has a competitive ratio 
at least -§y against an adaptive- online adversary. 

Basic idea. We fix an online memoryless randomized algorithm A. At the beginning A is given 
n + 1 items: a , a 1 , . . . , a n , where a = 1 + — . We consider n + 1 strategies for an adversary. The 
k-th. (0 < k < n) strategy is as follows: in each step collect a k , delete items a , a 1 , ...,a k , and issue 
their new copies. If A collected a 3 for j > k, issue a copy of a 3 as well. This way, in each step 
exactly one copy of each item is pending for A, while the adversary accumulates copies of the items 
a 3 for j > k. Since A is memoryless, in each step it uses the same probability distribution ((/j)j =0 , 
where qj is the probability of collecting a 3 . 

This step is repeated T S> n times, and after the last step both the adversary and A collect 
all their pending items. Since T n, we may focus only on the expected amortized profit in a 
single step, which is a k + Y!i>k 1i a% f° r the adversary and J2i 1i al f° r A. By solving a set of linear 
equations, we show (non-constructively) that for any probability distribution (qj)j, there exists k, 
for which the ratio between these gains is at least e/(e — 1). 
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6 Final Comments 



We provided upper and lower bounds for the competitive ratio of several variants of the item 
selection problem. The most intriguing open problems are to establish better bounds for (1) 
dynamic queue case and nontrivial bounds for (2) general randomized case. For (1), we have shown 
a lower bound of ~ 1.63 (improving the lower bound of (ft), but no upper bound better than 2 is 
known. (Better bounds are known for packet scheduling [12}[6]. but these algorithm use information 
about packet deadlines and do not seem to apply to dynamic queues.) We have also shown better 
bounds for some restricted cases: 1.8 for the FIFO queue (a generalization of packet scheduling 
with agreeable deadlines) and ~ 1.737 for the decremental queue. For (2), we have an e/(e — 1) 
lower bound, and we have shown it is tight in the uniform decremental case. 
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A The Uniform Case of Decremental Sets: Randomized Algo- 
rithms against Oblivious Adversaries 



In this part of appendix, we prove Theorems [2] and El i.e. we show that for the uniform case of 
decremental sets: 

(i) the competitive ratio of any randomized algorithm is at least e/(e — 1), 

(ii) the algorithm UniRand is e/(e — l)-competitive. 

Obviously, the lower bound holds also for non-uniform cases as well. 



A.l Lower Bound 

Fix any set size n and let 1Z = ^xj • jdrj"- The proof is based on Yao's min-max principle, i.e. we 
construct a probability distribution ir over inputs, such that (i) the gain of an optimal solution is 
n on any input sequence from the support of ir, and (ii) the expected gain of any deterministic 
algorithm run on a randomly chosen input from ir is at most n/lZ. 

We construct the distribution tt implicitly by the following random process: at each step, choose 
uniformly at random any active item a, collect it, and delete it after the step. Obviously, property 
(i) holds, and it is sufficient now to show (ii). 

Fix any deterministic algorithm A. Without loss of generality, A always collects an item if one 
is pending. Let E a<p be the expected number of items collected by A if a items are active and, 
among them, p items are pending. If p = 0, 1, then A collects p items. On the other hand, if p > 1, 
A collects an item in the first step, reducing the number of pending items to p — 1, and then, with 
probability {p — l)/a another pending element may get deleted. Hence, E a)P satisfies the following 
recurrence for any a > 1: E a> o = 0, E a> i = 1, and 

a-p+l p-\ 

&a,p — • &a-l,p-l H • &a-l,p-2 + -L 

a a 

for a > p > 2. 

The following technical lemma, can be used to bound the performance of any deterministic 
algorithm on tt. 

Lemma 13 For alla>p>0, we have E a , p < (a + 1)(1 - e~ p / a ) + 1. 

Proof: We prove the lemma by induction on p. The case p = holds trivially. For a > p = 1, 
E a , P = 1 < (a + 1)(1 - &- l/a ) + 1, as (a + 1)(1 - e" 1 ^) > 0. 

For a > p > 2, we use the inductive assumption and the recursive definition of E a ^ p , getting 

a-p+l p-1 

&a,p — • -&a-l,p-l H • Ai-l,p-2 + -L 

a a 
< (a - p + 1) (l - e~%=i^J + (p - l)a (l - e"^) + 2 

a — p 2a~p 

(a — p+ l)e a ( a - 1 ) + (p — l)e a ( a - 1 ) 



= a + 2-e~ p/a - 
< a + 2-e~ p/a - 
= (a + 1)(1 - e~ p / a ) + 1 



(a-p + l) (1+ a I ' )+(p_l) (l + 



a(a — 1) J \ a(a — 1) 
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where the second inequality follows from e x > 1 + x. This completes the inductive step and the 
proof of the lemma. □ 

By Lemma [131 we have that E n „ n , the expected number of items collected by A, is at most 
(n + l)(e — l)/e + 1 = n/lZ + 0(1). Thus, property (ii) holds and, applying Yao's principle and 
taking the limit on n, we obtain our lower bound. 

A. 2 Upper Bound 

In this part we show that the algorithm UniRand (which at each step collects one randomly chosen 
pending item) is e/(e — l)-competitive. 

The idea behind the competitive analysis is to prove first that the optimal strategy of the 
adversary can be easily described: without loss of generality, at each step the adversary collects 
one item and deletes the same item. Then we analyze the competitive ratio of UniRand against 
such an adversary. 

The number of items in the initial set is denoted by n. By a we denote the number of active 
items at a given step, and by p < a the number of pending items. Note that p is a random variable. 
We consider states of the game between UniRand and the adversary, conditioned on p being fixed, 
and we refer to such a state as a configuration (a,p). The definition of UniRand implies that in 
configuration (a,p) each active item is pending with probability p/a. 

Starting from each fixed configuration (a,p), we analyze the relation between the gain of UniRand 
and that of the adversary. Since the items are identical and Uni Rand's pending items are distributed 
uniformly, we need not specify which items are deleted by the adversary in a given step, only their 
number. 

We can split these steps into smaller parts: an elementary step consists in either collecting an 
item or deleting an item. We can thus describe the adversary's strategy as a sequence S E {t, d}*, 
where t means that the adversary collects an item (and allows UniRand to collect one as well, 
provided it has some pending items) and d means that the adversary deletes an item. In the 
following we consider only the feasible strategies S for configuration (a,p), i.e. those in which: (i) 
the number of d operations in S is a, (ii) in every suffix of S the number of t operations does not 
exceed the number of d operations. We call strategy S a k-strategy if it contains k t operations, 
i.e. the gain of the adversary on S is k. 

By E a ^ p [S] we denote the expected gain of Algorithm UniRand starting from configuration (a,p), 
versus adversary strategy S. 

The following lemma shows that the best strategy for the adversary, which guarantees that the 
adversary takes k elements, is (td) fc run on fc-element set. 

Lemma 14 Let S be a k-strategy for (a, a). Then E a ^ a [S] > E^^td)^. 

On the other hand for these simple adversary strategies, we may appropriately bound the E ajP 
values. 

Lemma 15 For any integers p < a, it holds that E aiP [(td) a ] > a(l — (1 — l/a) p ). 

While we prove the lemmas above in the following subsections, we argue now how they imply 
the competitiveness of UniRand. 
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Proof: [of Theorem [3] We fix any n-element set and any adversary's strategy S. Let k be the 
number of items collected by the adversary. By Lemmas [TH and [15j the gain of UniRand is E n<n [S] > 
^^[(td)^] > k{l — (1 — l/k) k ). Therefore, the competitive ratio is at most 



E n ,n[S] 



< 



< 



a 



A. 2.1 Relations between Adversary Strategies 



In this subsection, we show why (td) fc is the best fe-strategy. First, we prove two technical lemmas. 
Lemma 16 For any p < a and strategy S, it holds that E a ^ p+ i[S] > E a>p [S]. 

Proof: We prove the inequality by induction on pairs (p, S), where (p\, Si) < (p2, S2) if 1 5*1 1 < |<?2| 
or 1 5*i I = 1 and p\ < P2- The inductive basis is straightforward: if S = d a then E a „ + i [d a ] = = 
E a , p [d a }. Hp = 0, then clearly E a ,i[S) > = E afi [S]. 

In the inductive step, we have two cases depending on the first action of S. If S = tS', then 
E a , p +i[tS'] = 1 + E a ^[S'\ > 1 + E a>p -i[S'] = E a>p [tS'], by the inductive assumption. If S = dS", 
since we have p + 1 pending items, the adversary deletes a pending item with probability 2i- and 
a non-pending item with probability a ~^~ 1 . Using the inductive assumption, we get 

a — p — 1 p ~\~ 1 

E a [as'} = ^ — E a 

-i,p+i 

[ S >] + iL— Ea _ hp [S'] 



> £ -i, P [S'j + 

a 



—E a -\ P [S'} + —E a _i p -i[S'\ 
a a 



= E a>p [dS'} . 

□ 

Lemma 17 Let S = S\tdS2 be a feasible strategy for (a,p). If S' = SidtS^ is also a feasible 
strategy, then E ajP [S] > E a>p [S']. 

Proof: Suppose first that S = td$2- If p = 0, then the claim is obvious, as idols'] = E a fi[S'] = 0, 
and if p = 1, then E^ifS*] = 1 > E a \[S'\. So consider p > 1. By direct calculation and Lemma [TBI 
we get 

E atP [tdS 2 ] = l + E 0iP _i[dS 2 ] 

= H ^a-l^-ll^] H — £'a-l,p-2[5 , 2] 

= -E a -i p [tS2] + -E a ^i p ^i[tS2] 

a a 

= E a:P [dtS 2 ] • 
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Now consider the general case, when S = SitdSz and S' = S\dtS2- Then 

E a , P [S] = Ea tP [SitdS2] = ^ CjE a -k,p-i[tdS2] 

i 

E a , P [S'] = E ai p[SidtS2] = ^ CjE a -k,p-i[dtS2] 

i 

for some k, {cj}. By the previous calculations, it holds that E a -k jP -i[tdS2] > E a ^k,p-i[&'tS2] for 
all i, and hence 

Ea,p[S] = E at p[SitdS2] 

= ^ Q^a-fc,p-i[tdS 2 ] 

i 

> ^Q J B a _ fci p_ i [dt5 2 ] 

i 

= J E aiP [5idt5 2 ] 
= E a ^[S'} , 

completing the proof. □ 

Proof: [of Lemma [hi] We take any feasible A:-strategy S starting from configuration (a, a). It is 
straightforward, that we may iteratively apply Lemma[T7]and swap consecutive t and d operations, 
obtaining a feasible strategy d a " fc (td) fc at the end. Thus E a , a [S] > E a ^ a {d a ~ k (td) k ] = E k)k [(td) k ] 
□ 

A. 2. 2 Bound for the Best Adversary Strategy 

Proof:[of Lemma PT5] For p = 0, 1, E ajP [(td) a ] = p and the lemma trivially holds. Also if a = p = 2, 
then E a)P [{td) a ] = 3/2 = o(l - (1 - l/a) p ) . 

For the remaining values of a and p we prove the lemma by induction on a. We note that for 
P > 2 

E a:p [(td) a ] = a ~ P Q +l Ea-10-l [(td)"" 1 ] + ^£ a -l, p -2 [(td)"" 1 ] + 1 , 
as the pending items are distributed uniformly among active items. Note that this is the same 
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recurrence as in the proof of the lower bound. Thus, using the induction assumption, we get 
K )P [(td) a ] > V - (o-l) 1 



a- \ t 

(a - l) 2 (a-2\ p ~ 2 ( p-2 



1 + 



a \a — 1 J \ (a — l) 2 
a \a-ly V 



> a 

c 

'a- 1 



a 1 



completing the proof. □ 



B Deterministic Algorithms for Dynamic Queues 
B.l Lower Bounds 

Proof: [of Theorem[6] We now exhibit a sequence of 6 items for which Lemma[5]holds with 1Z ~ 1.63. 
To simplify notation, we rename the items: Z2, 24, zq, 23, z\ as x,y, z,u,v, respectively. Otherwise 
we follow the aforementioned idea. 

By Lemma O we want to find numbers x, y, z, u, v such that 0<z<y<u<x<v<l and a 
maximal 1Z for which: 

K-x < 1 
IZ-l < l + v 
K-(v + y) < l + v 

n -(i + u) < l + v + x + u 

7£ • + u + z) < 1 + v + x 

+ v + < l + v + x + u + z 

We can solve it by replacing inequalities by equations, and after doing substitutions, the problem 
reduces to finding a solution of a polynomial equation x 5 + x 4 + 5x 3 — x 2 — 1 = 0. This polynomial 
has exactly one real root, x = 0.61238..., which yields 1Z = 1.6329. 

Proof: [of Theorem [7] Fix a memoryless algorithm A. We give an adversary's strategy where the 
adversary's gain is 2 — o(l) times «4's gain. 

Pick large integers n and T S> re, and let X = {^o, x n } where w x . = 1 + ~ for i = 0, 1, n. 
The adversary maintains the invariant that at each step A's pending set is X, with the items 
ordered by increasing value. Suppose that for this pending set A collects some item x^. 
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If k = 0, the adversary collects item x n , deletes all items, inserts copies of all items from X 
again into the queue, and repeats the process T times. A J s gain is Tw xo = T while the optimum 
gain is Tw Xn = 2T, so the ratio is 2. 

Suppose now that k > 1. In this case, the adversary collects Xk-i, deletes all items xq, ...,Xk-i 
for i = 0,1, ...,k — 1 and inserts new copies of items xq, ...,X)-. This process is repeated T times. 
After T steps, the adversary collects the remaining uncollected items, in particular, all T copies of 
item x^. A can of course collect the remaining pending items. The value collected by A is at most 
Tw Xk + 2(n +1) = T(l + k/n) + 2(n + 1), while the value collected by the adversary is at least 
T(w Xk _ 1 + w Xk ) = T(2 + (2k — l)/n). So with T = n 3 and n — > oo the ratio approaches 2. □ 

B.2 Upper Bound for FIFO Queues 
Proof: [of Theorem [9] 

We define stages of the algorithm. The first stage begins before FIFOQueEH collects any item. 
Each next stage begins immediately after previous stage ends. The stage ends when h is deleted 
by the adversary or condition (H) holds. The last step of the stage is the last step t such that at 
beginning of t, b! was pending for FIFOQueEH. 

Let &\,e.i, . . . efc be the set of items collected (in this order) by FIFOQueEH in one stage when 
(E) condition held. Let h\,...,hk be the corresponding heavy items. From the algorithm and the 
definition of FIFO queues, we have: 

Fact 18 For all i = 1, k we have < hi (with all relations strict, except possibly for i = k) and 
h < h i+1 . 

The proof is by amortized analysis. We preserve the invariant that, after each stage, each 
item i that is pending for the adversary but has already been collected by FIFOQueEH has credit 
associated with it of value equal wi. The adversary's amortized gain is equal the his actual gain 
plus the total credit change. To prove the theorem, it is then sufficient to prove the following claim: 

(*) In each stage the adversary's amortized gain is at most 1.8 times FIFOQueEH 's gain. 

We prove (*) for each of the following cases (1) the stage ends because the heaviest item is 
deleted, or (2) the stage ends because FIFOQueEH collects h! . The second case has three sub-cases, 
depending on which condition the latest item a collected by the adversary satisfies: (2a) a <\ei for 
some / < k, (2b) ej~ < a < h/., or (2c) h^ < a. 

Case 1 : The stage ended because item h was deleted. The adversary cannot gain credit for any 
items taken by the algorithm, as they are not pending after that stage. Hence the gain of the 
adversary is at most J2i=i w hi- The gain of the algorithm is Yli=i w e r As w ei > f3wh t for all i, the 
competitive ratio is at most 4 = 1.5. 

Case 2 : The stage ends, because awh k+1 < Wh k and FIFOQueEH collected h].. 
Case 2a : Suppose that a <d ej for some I < k, choose minimal such I. Let Cj = w ei — Pw^ for 
i = 1, ... ,1—1. We estimate the amortized adversary's gain: in step i < I — 1 the heaviest pending 
item is hi, and in step i G {I . . . k + 1} the adversary collects item preceding e/. If this item is still 
pending for the algorithm, its weight is at most (3wh r If it is not pending for the algorithm, it is 
one of the e$'s, for some i < I. Then the gain of the adversary is w ei = 011)^ + Ci < f3wh, + c%. As 
we can collect each such e, only once, all these steps add to at most (k — I + 2)f3wf H + J2i=\ c %- The 
adversary gets credit for all items taken by the algorithm in steps I, . . . , k+1 that is Ym=i w ei + w h k - 
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Thus the amortized gain of the adversary is at most 

l-l l-l k 

^2w hx + (k-l + 2)(3w hl + ^2 a + ^2 w a + w h h , 

i=l i=l i=l 

On the other hand the gain of the algorithm is 

k i-i k 

Yl We * + Wh k = ^2(P w hi + Ci) + Yl We i + Wh k ■ 

i=l i=l i=l 

Hence for fixed I, the competitive ratio of FIFOQueEH in the stage is at most 

Et! h i + (k-l + 2)Pht + Et! Ci + Eli *i + h k 



n 



i,i 



E'=l PtH + E!=i a + E"=i e i + h k 



Fact 19 For any j,x,m > and n > x, either — < - or n x > — . 

We upper-bound Hij- All the following inequalities follow from Fact [19] (for 7 = 1 or 7 = 

^ . E!=l fei + (fe - j + 2)^j + EL e t + /t fe 
1 i S jzi z 

E i= i + Ei=« a + h k 

< Ej=! h i + (k-l + 2)f3h + Etj + h k 

Eti^ + EL^ + ^ 

< (fc-Z_+ 2)/3fc, + Etz 0fc + h 



1 + 



E?=j (3hi + h k 
(k-l + 2)phi 
EtiPh l + h k ' 



Fix fc, I and /i;. The fraction above is maximal when hi+i, . . . h k are minimal. As a/ij+i > hi, 
minimal hi,...h k form a geometric progression with a common ratio of a -1 . Thus by taking 
hi = a k ~ l h k for z > I and fixing m := k — I + 1 > 1, we obtain 



(k-l + 2)ph t im + l)(3a m ^ 1 
Tin < 1 + ^-r — - < 1 + r^- = 



For m = 1 we have Bi = 1 + 2/3/(1 + /?) = 1.8, and for m = 2 we have B 2 = l + 3/3a/(/?(l + a) + l) 
1 + 9/13 < 1.8. As for m > 3, we show that S m < B m -\: 



(m + l)/?^" 1 (m + l^c/™" 1 



B m - 1 = , — ■ < 



m + 1 m/3a m ~ 2 m/3a m ~ 2 

= a • • r; < ^ = Bm-1 — 1 ■ 

Thus all 1Z\ t i are upper-bounded by 1.8. 

Case 2b : Suppose e& <! a <\ h k . The adversary collected only the items preceding h k in the queue, 
thus gaining at most h\ + . . . + h k + 2h k , as in step z G {1 . . . k — 1} it can collect item of weight 
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greater than hi, in steps k, k + 1 it can collect items with weight almost hk and gain credit for hk- 
Hence the competitive ratio of FIFOQueEH can be upper-bounded by 



n ^ = E»=i hj + 2h k < E/=i h i + 2/j -fc = fi 

and by Fact 1191 for 7 = /3 either B < ^ = 1.5 or 

„ /u. + 2hb 3 
- /3% + /i fc 1 + 

Case 2c : Suppose % < a. Then he cannot get credit for any item taken by the algorithm, due to 
the EEF property. In this case the adversary's gain is at most hi + . . . + /ifc+i, which gives smaller 
ratio than the one obtained in case (2b), as a > ^. This concludes the proof. □ 

Proof: [of Theorem [10] Consider two instances. In the first one, we have a <\ b <\ c < d, with 
w a = Wb = (3 — e, w c = (3, Wd = 1. Item a is deleted right after the first step, and item b right 
after the second step. FIFOQueEH collects c in the first step and d in the second step, while the 
adversary collects all four items. Adversary's gain is 1 + 3/3 — 2e and the FIFOQueEH 's gain is 1 + /3. 
Thus the competitive ratio is arbitrarily close to (3/3 + l)/(/3 + 1). 

In the second instance, a <\ b < c <\ d, with w a = f3, Wb = w c = 1 — e, Wd = 1. Items a, b and c 
are deleted right after the second step. FIFOQueEH collects a in the first step and d in the second 
step, so its gain is 1 + (3. The adversary collects 6, c, d (in this order), so his gain is 3 — 2e. Thus 
the competitive ratio is arbitrarily close to 3/(1 + 13). 

From these two instances, we get that the ratio of FIFOQueEH is at least max {3/3 + 1,3}/ (/3+ 1), 
and this quantity is at least 1.8 for any (3. □ 

B.3 Upper Bound for Non-Decreasing Weights 

Set dominance relation. Let X, Y be two finite sets of numbers. We say that X dominates Y, 
denoted X >z Y, if either Y = or maxX > maxY and {X — maxJ) >z (Y — maxY). Note that 
we do not require that \X\ = \Y\. In particular, X y 0, for any X. 

For any set T and a number u, let $ U {T) = \{t G T:t > u}\. We show that the majorization 
can be described in terms of jj u The following lemma is routine and we omit the proof. 

Lemma 20 The following three conditions are equivalent: 

(i) XhY, 

(ii) There is an injection f :Y — > X such that f(y) > y for all y. 
(Hi) For every x we have %c(X} > Ux(Y). 

Lemma 21 Suppose that X y Y / 0. Then 

(i) X — va.m.X y Y — min Y . 

(ii) If x G X n Y then X - x ^ Y - x. 
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(Hi) If X,Y C Z , y E Z — Y, and x > max {z £ Z — X : z < y} then XUx yYUy. (In particular, 
this holds for x > y.) 

Proof: Parts (i) and (ii) are straightforward, so we only show (hi). Let x' = max {z £ Z — X : z < y} 
It is sufficient to show (hi) for x = x' . 

We use Lemma dpi). For u < x', since X y Y, we have | u (IUx') = % U (X) + 1 > j} u (y) + 1 = 
# u (y U y). For u > y, we have ft„(X U x') = $ U (X) > $ U (Y) = $ U {Y U y). Suppose x' < u < y. 
Since y £ Z — Y and X n [u,y] = Z n [u, y], we have |A n [tt, y]| > |y n [u, y]\, and therefore 
tJ u (X U x') = > UY) + 1 = k(Y U y). □ 

For simplicity, we assume that all weights are different. If there are equal weights, we can 
perturb them slightly or extend the weight ordering using the item indices. For sets B and C of 
items, we say that B dominates C, writing B y C, if {wb '■ b £ B} dominates {w c : c £ C}. We write 
B y aC if {wb : b £ B} dominates {aw c : c G C}. 

Assume that the adversary has EEF property. Consider the behavior of the adversary. By the 
EEF property, for any i, j S S, we can assume that if j is the item collected in step t and i<j then 
the adversary will not collect item i in the future. Thus, instead of considering the whole set of 
pending adversary items, we can restrict ourselves only to those that are after the one he collected 
last. Let Ct be the set of these items. We update Ct as follows: at each step, the deleted items are 
removed from Ct and released items are added to Ct- Then, if the adversary collects an item j, we 
remove all items i < j from Ct- 

To organize the accounting so that we can pay for these extra items, we do two things. One, we 
do not immediately give MarkAndPick credit for the collected items. We give it credit for collecting 
i only at the time when the adversary collects an item at least as heavy as i. This is formalized 
below in Lemma [22] where the collected items i that are heavier than the maximum adversary 
item contribute to the both sides of invariant (a) (they get included in Lt and L' t ), and only when 
the adversary collects an item greater than i, item i will be removed from L' t and contribute to 
preserving the invariant. 

Next, we keep track of the adversary's extra items so that we can pay for them later. When 
the adversary collects an extra item j and we mark h, we know that Wj < Wh/4>- We store j in a 
separate set E' and h in M' . Later, when there is an idle step at which the adversary collects h, 
since h has already been collected by the algorithm, its budget 4>Wh pays both for Wh (for the mark 
on h) and for Wj. Once the adversary "consumed" the extra step by collecting h, we move j to the 
set E of the extra adversary items that we already paid for. 

We represent our invariants in terms of the domination relations between some varying sets of 
items. The reason for this is that items can be added and removed from these sets in this process, 
and it does not seem possible to maintain appropriate bounds only between the total weights of 
these sets. 

Notation: Symbols Dt, Mt, and Lt represent respectively the sets of items collected by the 
adversary, marked by the algorithm, and collected by Algorithm MarkAndPick up to and including 
step t. L' t = Lt H Ct is the set of algorithm's items that the adversary may still collect in the 
future. Let also et = \Dt\ — \ Mt\ and It = \L' t \. Figure Q] illustrates this notation, as well as the sets 
introduced in the lemma below. 

Lemma 22 For each time step t, there exist disjoint sets Et,E[ C Dt with \Et\ = et and \E' t \ = it, 
and a set M[ C Mt with \M' t \ = It, such that 
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weight order j 

(max. adversary 
item) 



Figure 1: Notation. 

(a) cj>w(L t - L' t ) > w(M t - M' t ) + w(E t ), 

(b) (j)L' t h Mi h L' t , 

(c) M t h (A -Et- E' t ) U L' t , and 

(d) M>ycf>E>. 

Proof: We show that the invariant in the lemma is preserved. For simplicity, we omit the subscript t 
and write D = D u M = M t , etc. Also, let Aw(D) = w(D t+1 )-w(D t ), Aw(M) = w(M t+1 )-w(M t ), 
and so on. 

We view the process as follows: at each step, 

(I) The adversary first inserts items into 5, 

(II) then he selects the item j to be collected, 

(III) next, the adversary deletes some items from S (of course, only the items that are before j in 
S can be deleted), 

(IV) finally, both the adversary and the algorithm collect their items. 
In order to show (a), we need to show that 

(j)Aw(L) + Aw(M') > cf)Aw(L') + Aw(M) + Aw(E). (3) 

In addition, we need to show that (b), (c) and (d) are preserved. 

We look at all sub-steps separately. (I) Insertions do not affect the invariants. (None of the 
sets M, D, L, L' changes, and we do not change sets M' , E, and E' .) In (II), suppose the 
adversary selects j and f was an item collected by the adversary in the previous step. There 
may have been some items i, j' < i < j, that were in V . Since these items are now removed 
from C, they are also removed from L' , and we need to update M' and E' so that they have 
the same cardinality as L' , and in such a way that the invariants are preserved. Let i G L' be 
such an item with minimum weight, g the minimum- weight item in M' and e the minimum- weight 
item in E' . We remove i from L' , g from M' and e from E' . Since (j)Wi > w g , by (b), we have 
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4>Aw{L) + Aw(M') = 0-w g > -cj)Wi + + = 4>Aw{L') + Aw(M) + Aw(E), so (a) is preserved. 
Invariants (b) and (d) are preserved because we remove the minimum items from L', M' , and E' . 
In (c) , the left-hand side does not change and on the right-hand side we remove i from V and add e 
to D — E — E' , and by (b) and (d) we have wi > w g /4> > w e , so the right-hand side cannot increase. 
In sub-step (III), deletions do not affect the invariants. 

The rest of the proof is devoted to sub-step (IV). We examine ([3]) and the changes in (b), (c) 
and (d) due to the algorithm and the adversary collecting their items. 

Case A : There is at least one pending item. The algorithm marks h and collects the earliest pending 
item i such that Wi > Wh/ffi- Thus h is added to M and i is added to L. We do not change E. We 
have some sub-cases. 

Case A.l : i <j. Then i is not added to L' , and we do not change E' . Since (j)Aw(L) = <j)Wi > Wh = 
Aw(M) and Aw(E) = 0, to prove ([3]) it is now sufficient to show that 

Aw(M') > <pAw(L'). (4) 

If j ^ L (note that this included the case i = j), then V does not change and we do not change 
M', so is trivial. In (b) and (d) nothing changes. We add the maximum unmarked item h to 
M and j to D, so (c) follows from Lemma [2TY iii) . 

If i 7^ j and j £ L and then let g be the minimum item in M' and e the minimum item in E' . 
Item j is removed from V and we remove g from M' and e from E' . Since, by (b), (ftuij > w g , we 
have Aw(M') = —w g > —<fyujj = ipAw(L'), so (jl]) holds. Since j, g and e are minimal, (b) and (d) 
are preserved. In (c), moving j from V to D does not change the right-hand side. We also add h 
to M and e to D — E — E' U L' , so (c) is preserved because of the choice of h and Lemma [2lTiii) . 

Case A. 2 : it>j and j £ L. Then i is added to L' . We also add j to E'. Since Aw(L) = Aw(L') = Wi 
and Aw{M) = Wh, to show ([3]) it is sufficient to show that 

Aw(M') > w h . (5) 

Since \L'\ increased, we need to add one item / to M'. We choose this / as follows: If i < h, we 
choose f = h. Otherwise, if i > h then, by the choice of h, we get that all items h < / < i are 
now marked. In this case, we choose the largest / < i such that / ^ M' and add it to M'. Since 
h G M — M', h itself is a candidate for /, so we have h< f <i. 

Note that, in this case, by the choice of i (as the earliest pending item with weight at least 
Wh/'P)^ j O i and j ^ L, we have Wf > Wh > (f>Wj. In particular, this means that j <h and that all 
items h<\ f <]i are in L'. 

Since Wf > Wh, (O is trivial. Invariant (d) is also quite easy, since Wf > </>Wj, by the previous 
paragraph. In (c), adding j to D and £" does not change the right-hand side. We also add h to M 
and i to L', which preserves (c) by the choice of h and Lemma [2lTiii). 

To show (b), if i < /i then f = h and, since 0tUj > w/j > Wi, invariant (b) is preserved. If i l> h, 
then, (^Wj > Wi > Wf, so the first part of (b) is preserved. That the second part of (b) is preserved 
follows from the choice of / and Lemma l2lTiii) . 

Case A. 3 : i > j and j £ L. Then we remove j from L' and add i. We thus have Aw{L) = Wi, 
Aw(L') = wi — Wj and Aw(M) = Wh- Thus to show ([3]) it is sufficient to show that 

Aw(M') + (j)Wj > w h . (6) 
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We do not change E'. To update M', we proceed as follows. Let g be the lightest item in M'. 
Since j is the minimal element of V ', (b) implies Wj < w g < <pwj. We first remove g from M'. 
Next, we proceed similarly as in the previous case, looking for an item / that we can add to M 1 to 
compensate for removing g (since \M'\ cannot change in this case.) Let h' = max(<7, h). Note that 
h! G M — M' . If i < h' , we choose / = h'. Otherwise, if i > h 1 then, by the choice of h', we get that 
all active items b! < / < i are marked. In this case, we choose the largest / < i such that / ^ M' 
and add it to M'. Since h! G M — M', b! itself is a candidate for /, so we have b! < / < i. 

Note that, in this case, by j < g, all items h' < f <i are active and, by the choice of i, they are 
all in V. 

Now, in © we have Aw(M') + (f)Wj = {—w g + Wf) + 4>Wj > Wf > wy > Wh- In (d), the left-hand 
side can only increase (since / > g) and the right-hand side does not change. In (c), moving j from 
V to D does not change the right-hand side. We also added h to the left-hand side and i to V on 
the right-hand side, so (c) is preserved by the choice of h and Lemma [2lTiii) . 

In (b), removing j from L' and g from M' does not affect the invariant. Then we add / to 
M 1 and i to L'. By the algorithm, we have 4>Wi > Wh, while by the case assumption and (b), we 
have <j)Wi > cpWj > w g . Therefore (ftwi > wy. Since either / = h! or / < i, this implies 0u>, > Wf, 
showing that the first inequality in (b) is preserved. The second part of (b) follows again from the 
choice of / and Lemma [2TT hi). 

Case B : There are no pending items for the algorithm. It means that L' contains all active items 
i ^ J) including j. By the weight ordering assumption and the second part of (b) this implies that 
L' = M 1 . Since the adversary collects an item and the algorithm does not, e = \D\ — \M\ increases 
by 1, so we also need to add an item to E. Let b be the minimum-weight item in E'. We do this: 
we remove j from M' and from L' and we move b from E' to E. Using the choice of b and (d) we 
have Wj > 4>Wb, so 

4>Aw(L) + Aw(M') = + {-wj) = -4>Wj + Wj/4> > 

> -cjywj + + w b = (f)Aw(L') + Aw(M) + Aw(E), 

and thus (J3]) holds. By the choice of j and e as the minimum items in L' and E', respectively, 
invariants (b) and (d) are preserved. In (c), j moves from L' to D, and b moves from E' to E, so 
the right-hand side does not change. □ 

Proof: [of Theorem [TT] By the invariants of Lemma [22l at each time step it holds that 

(f>w(L t ) > [<Pw{L' t )-w{M' t )]+w{M t ) + w{E t ) 

> + HA -Et- E[) + w(L' t )} + w(E t ) 
= w{D t ) + w{L' t ) - w{E[) 

> w(D t ) + W (Ml)/ct>-w(E' t ) 

> w(D t ) , 
and the (^-competitiveness follows. □ 



C Randomized Algorithms for Dynamic Queues 

Proof: [of Theorem [12] Fix some online memoryless randomized algorithm A. Recall that by a 
memoryless algorithm we mean an algorithm that makes a decision on which item to collect based 
only on the weights of the pending items. 
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We consider the following scheme. Let a > 1 be a constant which we specify later and n be a 
fixed integer. At the beginning, the adversary inserts items a , a 1 , . . . , a n . (To simplify notation, 
in this proof we identify items with their weights.) In our construction we assure that in each step, 
the list of items which are pending for A is equal to (a , a , . . . , a n ). Since A is memoryless, in each 
step it uses the same probability distribution (qj)" =0 , where qj is the probability of collecting item 
a 3 . As the algorithm always makes a move, YH=o 9« = 

We consider n+1 strategies for an adversary, numbered 0,1, ... ,n. The fc-th strategy is as 
follows: in each step collect a k , delete all items a , a 1 , a k , and then issue new copies of all these 
items. Additionally, if the algorithm collected a 3 for some j > k, then the adversary issues a new 
copy of a 3 as well. This way, in each step exactly one copy of each a 3 is pending for the algorithm, 
while the adversary accumulates copies of the items a 3 for j > k. 

This step is repeated T ^> n times, and after the last step the adversary collects all uncollected 
items. Since T > n, we only need to focus on the expected amortized profits in a single step. 

We look at the gains of A and the adversary in a single step. If the adversary chooses strategy 
k, then it gains a k . Additionally, at the end it collects the item collected by the algorithm if this 
item is greater than a k . Thus, its amortized expected gain in single step is a k + X^fc?* *- The 
expected gain of A is ^ qia 1 . 

For any probability distribution (q^ )^=o °f the algorithm, the adversary chooses a strategy k 
which maximizes the competitive ratio. Thus, the competitive ratio of A is is at least 



for any coefficients vq, vj~ > such that v k = 1. Note that the latter term corresponds to the 
ratio forced by a randomized adversary who chooses k with probability v^. In particular, we may 
choose Vk to be the value for which the competitive ratio of such a randomized adversary strategy 
against any deterministic algorithm is the same. After solving the set of equations we get 





where M = a 



n+1 




For these values of Vk we get 



M1Z £\ qja 3 > Mv k a k + ^ Mv k ^ qj a j 



k k j>k 

n-1 



= ^ Mv k a k + Mv n a n + ^ qja 3 ^ Mv, 



'k = a 



n+1 



k=0 j k<j 



j 



Therefore, 1Z > 




. This bound is maximized for a = 1 + 1 /n, for which 




which tends to as n 



oo. □ 
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